Numerical analysis of shell-based geometric image inpainting algorithms and their semi-implicit extension

In this paper we study a class of fast geometric image inpainting methods based on the idea of filling the inpainting domain in successive shells from its boundary inwards. Image pixels are filled by assigning them a color equal to a weighted average of their already filled neighbors. However, there is flexibility in terms of the order in which pixels are filled, the weights used for averaging, and the neighborhood that is averaged over. Varying these degrees of freedom leads to different algorithms, and indeed the literature contains several methods falling into this general class. All of them are very fast, but at the same time all of them leave undesirable artifacts such as “kinking” (bending) or blurring of extrapolated isophotes. Our objective in this paper is to build a theoretical model, based on a continuum limit and a connection to stopped random walks, in order to understand why these artifacts occur and what, if anything, can be done about them. At the same time, we consider a semi-implicit extension in which pixels in a given shell are solved for simultaneously by solving a linear system. We prove (within CBS acknowledges support from the Leverhulme Trust project Breaking the non-convexity barrier, the EPSRC grant EP/M00483X/1, EPSRC centre EP/N014588/1, the Cantab Capital Institute for the Mathematics of Information, the CHiPS (Horizon 2020 RISE project grant), the Global Alliance project ’Statistical and Mathematical Theory of Imaging’ and the Alan Turing Institute. RH and TH acknowledge support from the EPSRC Cambridge Centre for Analysis and the Cambridge Trust. L. Robert Hocking Department for Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge CB3 0WA, UK Tel.: +44 1223 764284 E-mail: [email protected] Thomas Holding Mathematics Institute, Zeeman Building, University of Warwick, Coventry CV4 7AL Tel.: +44 (0)24 765 73495 E-mail: [email protected] Carola-Bibiane Schönlieb Department for Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge CB3 0WA, UK Tel.: +44 1223 764251 E-mail: [email protected] ar X iv :1 70 7. 09 71 3v 1 [ m at h. N A ] 3 1 Ju l 2 01 7 2 L. Robert Hocking, Thomas Holding, and Carola-Bibiane Schönlieb the continuum limit) that this extension is able to completely eliminate kinking artifacts, which we also prove must always be present in the direct method. Although our analysis makes the strong assumption of a square inpainting domain, it makes weak smoothness assumptions and is thus applicable to the low regularity inherent in images.